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Om  vogelaanvaringen  tijdens  oefenvluchten  te  voorkomen,  is  door  TNO-FEL  voor 
de  KLu  het  ROBIN  systeem  ontwikkeld.  Het  ROBIN  systeem  werkt  op  stilstaande 
radarbeelden.  Omdat  juist  beweging  van  objecten  in  het  radarbeeld  veel  informatie 
geeft  omtrent  de  aard  van  het  object,  biedt  bewegingsanalyse  op  een  reeks  beelden 
een  goede  mogelijkheid  om  filtertechnieken  te  ontwikkelen  die  beter  zijn  dan 
filtertechnieken  op  een  enkel  beeld. 

In  vervolg  op  ROBIN  is  daarom  het  project  "Bewegingsanalyse  vogeltrek"  opgezet 
met  als  doel  te  komen  tot  een  implementeerbaar  filter  gebaseerd  op 
bewegingsanalyse. 

In  het  onderzoek  wordt  een  aantal  mogelijke  methoden  van  bewegingsanalyse 
vergeleken.  Van  de  meestbelovende  "object  matching"  methode  wordt  een 
algoritme  ontwikkeld  en  een  aantal  implementatiefacetten  belicht.  De  bevindingen 
van  het  onderzoek  zijn  in  dit  rapport  opgenomen.  Verder  werd  een  implementatie 
gebaseerd  op  dit  onderzoek  gerealiseerd  en  aan  de  Klu  aangeboden. 

Een  uitgebreide  versie  ervan  is  thans  bij  de  Klu  in  gebruik. 
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1  Introduction 

1.1  Bird  migration 

In  the  lower  parts  of  the  airspace,  all  sorts  of  bird  movements  take  place. 
Depending  on  season  and  location,  these  bird  movements  can  either  be  structured 
or  random.  During  the  bird  migration  season  structure  can  be  recognized  in  the 
bird  movements.  Outside  the  bird  migration  seasons  the  bird  movements  are 
mostly  random,  although  there  can  be  structured  bird  movements  between  feeding 
areas  and  sleeping  areas. 

In  general  the  bird  movements  will  be  a  mixture  of  the  movements  of  several 
species.  Different  species  may  have  different  flight  routes,  velocities  and 
directions. 


1.2  The  ROBIN  system 

In  order  to  prevent  bird  strike  accidents  during  training  flights  of  the  Royal 
Netherlands  Air  Force  (RNLAF),  the  TNO  Physics  and  Electronics  Laboratory 
(TNO-FEL)  developed  the  ROBIN  system.  ROBIN  is  an  acronym  for  Radar 
Observation  of  Bird  INtensity.  Currently  ROBIN  is  being  used  operationally  by  the 
RNLAF. 

Using  a  radar  image,  ROBIN  estimates  the  bird  density  within  a  selected  region. 
The  estimation  is  used  to  decide  whether  -low  altitude-  flights  are  allowed  in  the 
selected  region.  The  images  that  ROBIN  uses  are  obtained  by  digitizing  the  video 
signal  of  an  Air  Traffic  Control  (ATC)  radar. 

The  current  ROBIN  system  operates  on  a  single  image  or  a  summation  of  images. 
In  the  case  of  a  single  image,  the  system  estimates  the  areas  in  which  there  are 
unwanted  image  elements  (e.g.  clutter).  These  elements  are  filtered  out  of  the 
image,  so  that  only  the  bird  echos  remain.  The  problem  is  to  distinguish  between 
clutter  and  bird  echos.  It  is  not  possible  to  get  a  perfect  separation  of  clutter  and 
bird  echos,  so  there  will  be  cases  where  bird  echos  are  missed  and  cases  where 
clutter  is  classified  as  bird  echo. 

In  the  case  of  a  summation  of  images,  we  can  consider  the  resulting  image  as  a 
time  exposure  of  the  radar  screen.  Moving  birds  can  be  identified  by  an  operator 
because  they  are  visible  as  tracks  in  the  summation  image. 
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1.3  Objectives  of  the  project  "Motion  Analysis  of  Bird  Migration” 

The  operational  ROBIN  system  uses  single  or  summed  radar  images.  In  the  case  of 
single  images,  no  information  about  the  movement  of  the  objects  in  the  image  can 
be  obtained. 

In  the  case  of  summed  images,  moving  objects  produce  tracks  of  echos  in  the 
images.  When  a  summation  image  is  viewed,  the  information  about  the  motion  of 
the  objects  is  limited  by  a  number  of  factors  : 

•  There  is  no  information  whether  an  object  moves  one  way  or  the  other. 

•  Only  spatial  dependencies  and  not  time-sequential  dependencies  between 
objects  are  visible,  which  cause  'false  tracks’  to  appear. 

•  In  the  case  of  a  higher  bird-density,  individual  echos  from  one  or  more  images 
will  aggregate. 

The  objective  of  the  project  "Motion  Analysis  of  Bird  Migration"  is  to  obtain 
motion  information  from  a  sequence  of  images  and  to  filter  images  using  that 
motion  information. 

An  other  way  of  obtaining  motion  information  would  be  by  using  a  tracking-radar 
or  by  using  the  Doppler  information  provided  by  some  radars.  These  possibilities 
will  not  be  investigated  here. 


1.4  Digital  radar  images 

ROBIN  operates  on  digital  radar  image.  A  digital  radar  image  consists  of  a  two 
dimensional  array  of  picture  elements,  better  known  as  pixels.  The  value  of  a  pixel 
is  directly  proportional  to  the  amplitude  of  the  analogue  video  signal  at  the 
corresponding  location  in  the  image.  Contrary  to  the  analogue  case  however,  only 
a  limited  number  of  values  (e.g.  255)  are  allowed. 

If  there  are  more  than  two  signal  levels  in  an  image,  we  speak  of  a  grey  value 
image.  If  there  are  only  two  possible  levels  (e.g.  0  and  1)  however,  we  speak  of  a 
binary  image.  In  binary  images,  the  two  possible  values  of  a  pixel  usually  indicate 
the  presence  or  absence  of  a  certain  property  at  the  location  of  the  pixel. 
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2  Characteristics  of  radar  images 


The  following  elements  can  be  part  of  a  radar  image  : 

•  birds  or  groups  of  birds 

•  aircraft  and  ships 

•  rain-,  sea-  and  ground  clutter 

•  noise  and  interference 

It  should  be  noted  that  not  all  of  these  elements  are  necessarily  present  in  one 
image.  In  the  following  paragraphs  the  properties  of  the  different  image  elements 
will  be  discussed.  Subsequently  we  will  discuss  the  possibilities  for  using  motion 
information  to  filter  undesirable  elements  from  the  images. 


2.1  Bird  echos 

Usually  a  bird  echo  in  the  image  will  result  from  a  group  of  birds.  An  individual 
bird  can  also  be  detected,  depending  on  distance  and  size  of  the  bird.  A  bird  echo 
in  the  digital  image  will  consist  of  a  spot  whose  size  will  vary  from  a  few  pixels  to 
a  few  dozens  of  pixels,  depending  on  the  number  and  size  of  the  birds.  Because  a 
bird  moves  its  wings,  its  relative  reflectivity  of  radar  waves  will  vary  with  time^. 

In  the  case  of  a  group  of  birds  this  effect  is  reduced,  provided  that  there  is  no 
correlation  between  the  wing  movements  of  the  birds  in  the  group. 

In  this  case  there  are  other  effects  however.  In  the  first  place  the  group  can  change 
its  shape  during  flight.  The  second  effect  is  that,  depending  on  the  wavelength  of 
the  radar,  interference  can  occur  between  the  waves  reflecting  from  different  birds 
in  the  group.  Both  effects  result  in  a  variation  in  shape  and  amplitude  of  the 
reflection,  which  can  even  cause  a  reflection  to  disappear  from  the  image. 

If  the  bird  density  is  very  high,  groups  of  birds  cannot  be  detected  individually. 
The  echos  from  different  groups  'melt'  together.  The  main  cause  is  that  the  current 
radar  does  not  provide  any  information  concerning  the  height  of  the  objects.  All 
objects  in  the  same  area  but  with  different  heights  are  imaged  on  the  same  location 
in  the  image. 


I 


L.  S.  Buurma  and  B.  Bruderer,  "The  application  of  radar  for  bird  strike 
reduction".  Compilation  for  the  Bird  Strike  Comittee  Europe,  The  Hague,  May 
1990 
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2,2  Echos  of  ships  and  aircraft 

Since  the  primary  use  of  the  radar  that  is  used  for  ROBIN,  is  the  detection  of 
aircraft,  it  is  obvious  that  aircraft  and  in  some  cases  ships  will  be  visible  in  the 
images.  These  objects  disturb  the  bird  density  estimation.  Therefore  it  is  important 
that  these  echos  can  be  removed  from  the  images. 

Since  aircraft  and  ships  are  metal  objects,  their  reflectivity  for  radar  waves  is  much 
larger  than  that  of  birds.  In  general  aircraft  and  ships  produce  strong  echos  that 
have  a  relatively  small  variation  in  their  appearance  and  that  have  a  steady  course. 
Aircraft  echos  move  faster  than  bird  echos. 


2.3  Clutter  echos 

2.3.1  Rain  clutter 

If  a  rain  cloud  is  dense  enough,  it  can  be  detected  by  radar.  Generally  the 
dimensions  of  a  cloud  echo  are  much  larger  than  that  of  a  group  of  birds.  A  cloud 
echo  can  even  fill  the  entire  image. 

A  cloud  can  consist  of  a  dense  area  with  sharply  defined  boundaries,  but  also  of 
large  areas  of  lower  density.  In  that  case,  it  depends  on  the  local  density  whether 
detection  by  radar  takes  place.  In  the  radar  image  this  results  in  a  large  area  with  a 
speckled  texture,  which  is  difficult  to  distinguish  from  a  concentration  of  bird 
echos. 


2.3.2  Land  clutter 

Bird  detection  takes  place  in  the  lower  air  layers.  One  of  the  consequences  is  that 
there  can  be  echos  in  the  radar  image  that  result  from  objects  on  the  ground  (trees, 
towers),  especially  in  the  region  near  the  radar. 

If  an  object  is  tall  enough,  it  will  certainly  be  detected  by  the  radar.  Beside  that, 
the  atmospheric  conditions  can  influence  the  propagation  of  the  radar  waves  in 
such  a  way  that  the  waves  bend  towards  the  earth  surface  and  reflect  from  it.  This 
can  result  in  an  image  filled  with  echos. 

This  disruption  of  the  radar  image  is  called  land  clutter.  Since  the  clutter  results 
from  objects  on  the  ground,  it  will  not  move  in  the  radar  image,  although  the  shape 
and  intensity  of  the  clutter  can  vary  in  time,  due  to  the  variation  in  propagation. 


2.4  Noise  and  interference 

Noise  occurs  in  the  signal  chain  of  the  radar.  The  noise  in  the  image  is  not 
correlated  in  space  and  time.  The  ROBIN  system  uses  an  active  control 
mechanism  that  adapts  the  gain  of  the  system  depending  on  the  amount  of  noise 
detected  in  the  signal.  This  causes  the  number  of  noise  pixels  in  an  image  to  be 
approximately  constant. 
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Interference  in  radar  images  is  often  caused  by  other  radars  of  which  the  waves 
interfere  with  the  bird  radar  or  by  side-lobes  of  the  bird  radar  itself. 


2.5  Filtering  using  motion  information 

2.5.1  Motion  information 

Besides  the  fact  that  motion  information  on  itself  can  be  important  for  bird  density 
forecasts,  it  can  also  be  used  to  distinguish  between  bird  echos  and  other  echos.  In 
this  way  radar  images  can  be  filtered,  which  results  in  a  better  estimation  of  bird 
density. 

2.5.2  Filtering  of  rain  clutter 

In  general,  rain  clouds  will  have  a  motion  pattern  that  differs  from  that  of 
migrating  birds.  This  means  that,  using  motion  information,  clouds  can  be 
distinguished  from  birds.  Motion  information  can  be  used  as  additional  criterion 
next  to  criteria  as  size  and  shape  of  the  echo.  It  is  even  possible  to  use 
meteorological  information  in  order  to  distinguish  clouds  from  birds. 

2.5.3  Filtering  of  land  clutter 

Land  clutter  can  be  considered  to  be  standing  still.  Depending  on  changing 
propagation  conditions,  the  amplitude  of  the  clutter  can  vary  with  time,  but  the 
location  of  the  clutter  remains  the  same.  This  property  can  be  used  in  detecting 
ground  clutter.  Since  we  are  interested  in  the  motion  of  birds,  it  is  possible  to  filter 
out  all  the  objects  that  have  a  velocity  lower  than  the  minimum  bird  velocity,  thus 
removing  ground  clutter. 

2.5.4  Filtering  of  noise 

The  noise  that  occurs  in  images  is  not  correlated  in  time  and  in  place.  This  means 
that  in  a  sequence  of  images,  the  noise  pixels  will  be  at  different  positions  in 
subsequent  images.  If  the  noise  pixels  are  considered  to  be  objects,  these  objects 
seem  to  move  randomly  without  any  coherence.  This  fact  can  be  used  to 
distinguish  noise  from  birds,  since  birds  will  tend  to  have  a  less  random  motion 
pattern. 
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3  Methods  of  motion  analysis 


There  are  three  methods  of  motion  analysis  that  will  be  compared  in  this  chapter, 
knowing  : 

•  the  optical  flow  method. 

•  Hough  transformation  of  summation  images. 

•  object  matching. 

The  specific  properties,  advantages  and  disadvantages  of  the  different  methods 
will  be  discussed.  One  of  the  methods  will  be  chosen  and  will  be  discussed  in 
further  detail  in  chapter  4. 


3.1  The  optical  flow  method 

The  optical  flow  method  is  based  on  the  flow  of  light  intensity.  An  image  is 
defined  as  a  two-dimensional  distribution  of  light.  If  we  have  a  sequence  of  images 
where  motion  is  present,  this  can  be  modelled  by  assigning  a  motion  vector  to  each 
of  the  pixels  in  each  image.  This  motion  vector  indicates  where  the  corresponding 
pixel  will  be  located  in  the  subsequent  image  in  the  sequence.  The  result  of  this 
method  consists  of  an  optical  flow  field  (Example  in  fig  3.1). 

3.1.1  Intensity  constraint 

The  optical  flow  method  is  based  on  the  conservation  of  intensity.  This  means  that 
no  intensity  is  lost  in  the  transition  from  one  image  to  the  other.  This  requires  that : 

I(x  +  Ax,  y  +  Ay,  t  +  At)  =  I(x,  y,  t)  (3. 1) 

where  x  and  y  are  image  coordinates  and  t  is  time.  We  call  this  condition  the 
intensity  constraint.  If  we  perform  a  Taylor  expansion  of  the  left  term  of  eq.  3.1, 
we  get : 

I(x  +  Ax,  y  -F  Ay,  t  +  At)  = 

I(x,  y,  t)  4-  dJ/dx  ■  Ax  +  dl/dy  ■  Ay  +  dl/dt  ■  At  -t- .  (3.2) 
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Fig.  3.1:  Example  of  an  optical  flow  field. 

From  3. 1  and  3.2  we  learn  that  if  we  let  At  ->  0  and  neglect  high  order  terms,  we 
can  say  that : 

Ix(x,  y)  •  «(x,  y)  +  Iy(x,  y)  •  v(x,  y)  +  It(x,  y)  =  0  (3.3) 

where  I^Cx.y) ,  Iy(x,y)  and  It(x,y)  are  the  gradients  of  intensity  :  dFdx,  3F3y  and 
3F3t  and  w(x,y)  and  v(x,y)  are  the  velocities  dx/dt  and  dy/dt  in  the  point  x,  y.  The 
three  intensity  gradients  can  be  determined  from  the  image  sequence,  while  «(x,  y) 
and  v(x,  y)  are  the  unknowns. 

Because  of  the  fact  that  we  have  two  unknowns  and  only  one  equation  for  each 
point  (x,  y),  we  will  need  an  extra  constraint  to  find  a  unique  solution. 

A  possible  extra  constraint  is  the  smoothness  constraint.  This  constraint 
corresponds  to  the  assumption  that  the  optical  flow  has  a  smooth  behaviour  and 
that  neighbouring  pixels  have  similar  motion  vectors. 


3.1.2  Calculating  the  optical  flow  field 

Finding  the  optical  flow  field  involves  the  minimization  of  a  functional  that 
depends  on  the  intensity  constraint  and  the  smoothness  constraint.  We  must 
minimize  this  functional  with  respect  to  the  unknowns  u  and  v.  For  our  type  of 
images  (radar  image  of  birds,  10  sec  between  images)  it  is  very  hard  to  determine 
the  gradients  of  intensity  in  (3.3)  because  the  movement  of  a  birdspot  is  several 
pixels  in  subsequent  images,  and  the  intensity  of  the  spot  is  not  constant. 
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3.2  The  Hough  transformation 

3.2.1  Applicability  of  the  Hough  transform 

The  Hough  transform  can  be  used  on  summation  images  and  on  lists  of  (x,  y) 
coordinates  that  give  the  positions  of  (centres  of  gravity  oO  objects.  ROBIN  has 
the  ability  to  create  an  image  that  is  the  summation  of  a  given  sequence  of  images. 
Birds  that  are  in  motion,  will  show  up  as  dotted  tracks  in  the  summation  image. 
For  a  human  being  it  is  not  hard  to  recognise  these  tracks,  even  if  the  dots  are 
somewhat  apart.  This  fact  is  used  in  the  current  method  of  performing  bird  counts, 
where  a  time  exposure  taken  from  the  radar  screen  is  interpreted  by  an  operator. 
To  achieve  track  recognition  by  the  computer,  we  need  to  have  some  sort  of 
structuring  transformation  that  can  be  applied  to  an  image.  The  Hough 
transformation  is  such  a  transformation.  This  transformation  registers  how  many 
points  in  a  summation  image  or  in  an  (x,  y)  list  satisfy  a  certain  equation.  If  this 
equation  represents  a  line,  it  can  therefore  be  used  to  count  the  number  of  points 
that  are  on  the  same  line.  If  this  number  exceeds  a  certain  threshold,  there  is 
enough  'evidence'  that  there  is  a  bird-track  present. 

3.2.2  The  principle  of  the  Hough  transformation 

The  Hough  transform  transforms  an  image  to  a  parameter  space.  To  describe  a 
line,  we  can  choose  from  different  representations.  One  of  these  representations  is 
the  normal  equation  :  y  -  ax  +  b.  Here  a  and  b  are  the  parameters  of  the  line.  This 
representation  however  has  one  disadvantage  :  a  line  parallel  to  the  y  -  axis  cannot 
be  represented. 

A  more  suitable  representation  therefore  would  be  :  x  cos  9  +  y  sin  0  =  p.  The 
meaning  of  0  and  p  are  illustrated  in  fig.  3.2.  The  Hough  transform  creates  a 
parameter  space  where  p  and  0  are  the  parameters.  A  straight  line  in  normal  space 
corresponds  with  one  single  point  (p,  0)  in  the  parameter  space. 


Fig.  3.2:  Parameters  p  and  9  of  a  line  in  the  .x,  y-plane. 

For  every  combination  of  p  and  0  there  is  one  unique  line  in  the  original  image.  A 
single  point  (Xp,  yp)  in  the  original  image  can  be  considered  to  be  part  of  an 
infinite  number  of  lines,  since  for  every  value  of  p  a  value  of  0  can  be  found  for 
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which  Xp  cos  9  +  Yp  sin  0  =  p  is  satisfied.  This  means  that  a  single  point  in  the 
original  image  transforms  into  a  sinusoid  in  the  Hough  space. 

Now  suppose  there  are  a  number  of  points  in  the  original  image  that  are  on  the  line 
given  by  :  X  cos  01  +  y  sin  01  =  Pi .  Each  of  these  points  produces  a  sinusoid  in  the 
Hough  space  (fig  3.3).  All  these  sinusoids  intersect  in  the  point  (pi,  9i).  By 
selecting  the  points  in  the  Hough  space  where  the  most  sinusoids  intersect,  the 
lines  on  which  the  most  points  are  situated  are  selected  in  the  original  image. 

The  implementation  of  the  Hough  transformation  can  be  divided  into  the  following 
steps; 

•  For  every  nonzero  pixel  in  the  summation  image  or  for  every  (x,  y)  pair  in  the 
coordinate  lists,  calculate  the  sinusoid  in  the  HOUGH  space. 

•  For  every  point  (p,  9)  in  the  Hough  space,  calculate  how  many  sinusoids 
intersect  in  the  point. 

•  Take  only  the  points  that  are  a  local  maximum  in  the  Hough  space. 

•  Generate  the  lines  that  correspond  to  the  points  (p|,  0;)  that  have  remained. 

An  example  of  the  use  of  the  Hough  transform  is  illustrated  in  fig.  3.3. 


Fig.  3.3:  E.tample  of  the  use  of  the  Hough  transform. 

The  result  of  the  Hough  transformation  is  a  set  of  lines  on  which  at  least  a  certain 
number  of  points  are  situated. 
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3.3  Object  matching 

3.3.1  Preprocessing  of  images 

The  method  is  based  on  the  assumption  that  an  image  is  composed  of  a  number  of 
distinguishable  objects  and  that  there  is  a  certain  amount  of  similarity  between  the 
objects  (in  subsequent  images)  that  result  from  the  same  source. 

It  is  necessary  to  perform  some  preprocessing  that  isolates  the  objects  and  that 
measures  a  number  of  parameters  that  are  characteristic  for  the  objects. 

3.3.2  Matching  process 

Any  sequence  of  objects,  obtained  from  different  images  in  the  right  order  of  time, 
forms  a  potential  bird  trajectory  or  path.  To  each  path  a  score  can  be  assigned, 
expressing  how  well  the  objects  in  the  path  fit  together.  The  combination  of  paths 
with  the  best  overall  score  is  the  result  of  the  matching  process. 

It  is  possible  that  there  are  objects  that  do  not  belong  to  any  path  or  on  the  other 
hand  that  a  path  misses  one  or  more  of  its  objects. 

3.3.3  Parameters  for  score  evaluation 

There  are  a  number  of  parameters  that  are  suitable  to  be  used  in  the  calculation  of 
the  score. 

a)  The  position  of  (the  centre  of  gravity  of)  an  object:  the  movement  within  a 
certain  time-interval  is  limited. 

b)  The  velocity  and  direction  of  an  object,  generally  these  will  vary  relatively  little 
in  a  short  period  of  time. 

c)  The  strength  of  the  signal  of  the  object.  Although  fluctuations  will  occur,  there 
will  be  some  correlation  in  time. 

d)  The  number  of  objects  that  are  part  of  a  path  (referred  to  as  the  length  of  the 
path).  If  many  objects  in  a  path  are  absent,  the  path  will  be  less  probable. 

e)  Collectivity  of  motion.  The  individual  (groups  of)  birds  of  the  same  species 
usually  travel  in  the  same  direction  at  the  same  speed. 

3.3.4  Optimization 

The  method  where  the  overall  score  is  optimized  as  function  of  all  the  possible 
combinations  of  paths,  as  described  in  par.  3.3.2,  will  need  a  large  amount  of 
calculations.  This  is  caused  by  the  large  number  of  possible  paths  and  the  even 
larger  number  of  combinations  of  paths. 

If  however  we  are  willing  to  accept  a  sub-optimal  solution  to  the  optimalization 
problem,  it  may  be  possible  to  reduce  the  amount  of  processing  required 
significantly. 
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3.4  Usefulness  of  the  methods 

3.4.1  The  optical  flow  method 

If  the  optical  flow  method,  using  the  smoothness  constraint,  is  applied  to  a 
sequence  of  images,  the  result  consists  of  a  two-dimensional  velocity  vector  field. 
Within  this  vector  field,  direction  and  amplitude  of  neighbouring  vectors  are 
similar.  There  are  no  discontinuities  in  the  field.  One  of  the  consequences  is  that  it 
is  not  possible  that  velocity  vectors  cross  each  other. 

The  main  disadvantage  of  this  method  becomes  clear  if  we  consider  the  fact  that  it 
is  possible  that  there  are  birds  with  different  directions  of  flight  in  one  radar  image 
(see  par  1.1).  The  optical  flow  method  cannot  model  this  situation,  because  of  the 
nature  of  its  result  as  described  in  the  above  paragraph.  Therefore  the  optical  flow 
method  is  inadequate  for  motion  analysis  on  bird  images. 

3.4.2  The  Hough  transformation 

The  result  of  a  Hough  transformation  consists  of  a  set  of  parameters  of  lines  on 
which  a  certain  number  of  points  lie.  The  transformation  does  not  provide  any 
information  about  the  exact  location  of  the  points  on  the  line  however.  This  means 
that  if  the  set  of  parameters  is  used  to  project  lines  onto  the  original  image,  it  is 
hard  to  say  where  the  lines  start  and  end. 

There  will  be  post-processing  involved  in  finding  out  what  parts  of  a  line  should 
remain  and  what  parts  can  be  discarded.  If  it  is  impossible  to  do  so,  the  image  will 
fill  up  with  lines,  which  does  not  provide  any  information.  Beside  that,  no 
information  can  be  provided  about  the  amplitude  of  the  velocity,  only  the  direction 
can  be  found. 

The  main  disadvantage  of  this  method  is  the  behaviour  at  high  bird  densities.  In 
that  case,  it  is  very  likely  to  find  a  number  of  points  that  look  as  if  they  lie  on  a 
straight  line,  even  if  no  such  dependency  exists.  It  has  become  impossible  to 
distinguish  between  real  lines  and  lines  resulting  from  points  that  form  lines 
randomly.  This  effect  occurs  because  no  time-dependency  is  used  in  this  method. 

3.4.3  Object  matching 

The  possibility  of  using  a  combination  of  the  parameters  described  in  par.  3.3.2  for 
the  score  function  is  very  attractive,  furthermore  paths  do  not  have  to  be  limited  to 
straight  lines.  Crossing  paths  are  no  problem  and  the  time-dependency  is 
incorporated  in  the  score  by  means  of  position  and  velocity.  The  method  offers 
possibilities  to  create  a  continuous  tracking  process  using  partially  overlapping 
subsequent  time-intervals.  To  limit  the  processing  time  required,  a  number  of 
'tricks'  will  be  needed. 
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3.4.4  Preliminary  conclusion 

The  optical  flow  method  is  not  suitable  in  the  case  of  superimposed  migration  of 
several  species  having  different  velocities  and  flight  directions.  The  Hough 
transform  ignores  the  coherence  in  time  and  position  of  consecutive  echoes  and 
does  not  discriminate  between  individual  tracks.  Object  matching  has  none  of 
these  disadvantages  and  merges  several  types  of  information. 

Therefore  object  matching  is  regarded  as  the  most  promising  method  for  motion 
analysis  of  bird  images.  This  method  is  chosen  for  further  investigation. 
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4  ALGORITHM  for  object  matching 


4.1  The  score  function 

4.1.1  Requirements  for  the  score  function 

The  score  function  determines  the  results  of  the  matching  process.  The  score  is 
some  kind  of  measure  for  the  probability  that  a  path  formed  by  a  sequence  of 
objects  is  a  real  bird  trajectory.  The  score  is  only  used  in  comparisons  between 
paths  and  does  not  need  to  have  any  physical  or  statistical  meaning. 

Because  the  score  function  has  to  be  evaluated  many  times,  the  function  must  not 
be  very  complicated.  Preferably  it  should  be  possible  to  use  partial  results  of  a  path 
for  the  score  calculation  of  an  extended  path.  The  different  criteria  should  be  easy 
to  combine,  for  example  as  a  weighed  sum. 

4.1.2  Description  of  the  score  function 

The  model  for  a  bird  trajectory  is  a  straight  line  with  constant  speed  and  identical 
echos  along  the  track;  the  speed,  flight  direction  and  echo  strenght  have 
disturbances  which  are  assumed  to  have  a  random  character  with  a  Gaussian 
distribution. 

The  formula: 


could  be  used  as  the  score  function,  where 


xj  is  any  of  speed,  direction  or  strength  of  the  objects 

pj  and  Oj  are  the  corresponding  mean  values  and  deviations 

n  is  the  total  number  of  parameters  for  all  objects. 

Parameters  which  are  incomparable  in  nature  are  transformed  to  indentical 
quantities.  Greater  differences  between  xj  and  pj  lead  to  a  lower  score.  However, 
longer  sequences  of  objects  will  also  lead  to  a  lower  score  even  if  the  differences 
Xj  -  pj  are  small. 

The  solution  to  this  can  be  found  when  the  random  character  of  the  xj  is 
considered. 


Let  Xj,  i  =  l..n  be  independent  random  variables  with  Gaussian  distribution. 
Then  the  formula 


x’-=Xf 


i=i 


is  also  random  following  the  chi-square  distribution  with  n  degrees  of  freedom. 
The  probability  that  is  less  than  a  certain  value  j}  is  given  by  the  function 
P(X^,  n).  In  the  reverse  case 
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^-(n,  P)  is  the  value  for  which  P  is  the  probability  that  is  greater  than 

zero. 

If  P  is  chosen  to  be  constant  then 

is  a  comparable  quantity  for  different  values  of  n. 

Some  values  of  %^(n,  P)  are  listed  in  table  4.1. 

Table  4.1 :  versus  P  and  n. 


p  = 

0.100 

0.500 

0.900 

n  =  1 

0.016 

0.455 

2.706 

2 

0.211 

1.386 

4.605 

3 

0.584 

2.366 

6.251 

5 

1.610 

4.351 

9.236 

7 

2.833 

6.346 

12.017 

10 

4.865 

9.342 

15.987 

15 

8.547 

14.339 

22.307 

20 

12.443 

19.337 

28.412 

30 

20.599 

29.336 

40.256 

Since  the  expectation  value  of  is  n,  the  choice  of  P  can  be  used  to  adjust  the 
balance  between  the  lenght  of  a  sequence  and  deviations  from  uniform  movement. 
Unfortunately,  the  values  for  li;  en  a;  are  not  known  and  the  speed  and  the  flight 
direction  are  not  easily  obtained  from  the  positions  in  polar  or  rectangular 
coordinates. 

A  guess  for  Oj  can  be  made  by  observing  bird  motion  when  the  bird  density  is  low 
and  the  tracks  can  not  be  confused. 

The  jij  are  estimated  from  the  object  data  of  the  current  path  for  which  the  score 
has  to  be  calculated;  the  effect  is  that  the  degrees  of  freedom  have  to  be  reduced. 
Finally  the  score  function  is  rewritten  in  terms  of  the  rectangular  velocity 
components  instead  of  speed  and  direction. 

The  formula  for  the  score  function  after  these  manipulations  is:(See  Appendix) 


score 


X(Vxr  +  Vy,’)- 
X-(3k-2.P)  -  - 


i  =  l 


i=l 


k.X(Vx,’+Vy,>)=  ,  (k  +  D.^m,-  J 

-  j(Vx2-+Vy.  =  )  - 2-3=8 - £ 


ITl: 


£(Vx,’-+Vy,8) 


i  =  I 


i  =  l 


I 

i=0 


m, 


i=0 
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This  function  is  easy  to  calculate;  if  the  search  over  all  paths  is  organised  as  depth- 
first  search  tree,  the  partial  sums  may  be  stored  in  a  stack  and  reused  for  all 
possible  extended  paths.  The  values  of  x~  are  calculated  once  and  stored  in  an 
array  (depending  on  k). 


4.2  Speeding  up  the  algorithm 

4.2.1  Sub-optimal  solution 


Fig  4.1:  Non  unique  solution. 

Optimization  over  all  combinations  of  paths  is  a  very  costly  computation,  because 
of  the  combinatorial  explosion.  If  the  paths  are  sufficiently  far  apart,  not  all 
possibilities  have  to  be  considered  and  local  optimization  is  possible. 

In  the  extreme  case,  optimization  can  be  performed  path-wise.  To  satisfy  the 
condition  that  an  object  can  only  be  part  of  one  path,  it  is  necessary  to  choose  only 
the  best  path  and  repeat  the  procedure  for  all  remaining  objects  until  no  more 
candidates  remain.  Fig  4. 1  illustrates  that  this  approach  does  not  always  lead  to  the 
best  solution. 

In  case  (a)  of  fig.  4. 1  the  bold  line  has  the  highest  score  of  all  possibilities  and  the 
line  is  composed  of  the  remaining  objects,  yielding  a  very  low  score. 

Case  (b)  of  fig.  4.1  shows  two  solutions  each  with  a  score  lower  than  the  best  of 
case  (a)  but  the  sum  of  the  scores  may  be  higher  then  in  case  (a). 
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4.2.2  Reducing  the  number  of  potential  paths 


□ 


Time  t 


Time  t  +  At 


Fig  4.2:  The  definition  of  Ap/At. 

Instead  of  using  the  entity  Ap/At  (as  depicted  in  fig  4.2)  in  the  score  function  for 
all  possible  paths,  only  those  pathsegments^  are  considered  for  which  Ap/At  is 
within  certain  limits.  A  side  effect  is  that  objects  that  are  very  fast  or  very  slow  can 
end  up  in  the  wrong  paths. 

This  mechanism  may  also  be  used  to  exclude  certain  speed/direction  combinations 
(rain  clutter  removal)  or  to  select  a  speed/direction  region  in  the  case  of  a  strong 
collective  motion. 

4.2.3  Using  sub-sequences  of  images 

In  the  case  of  longer  sequences  of  images,  the  number  of  potential  paths  grows 
roughly  with  [the  number  of  objects  per  image]  to  the  power  [the  number  of 
images],  even  if  the  restrictions  of  paragraph  4.2.2  are  applied. 

A  shorter  sequence  of  images  is  better  for  the  processing  time,  but  finding  the 
correct  matches  becomes  more  difficult  because  of  the  limited  amount  of 
information.  By  applying  repeated  operations  on  (overlapping)  partial  sequences, 
the  processing  time  for  long  sequences  can  be  diminished.  In  this  case  the  last 
segments  of  the  paths  found  in  a  previous  stage  are  used  as  'seeds'  for  paths  in  the 
new  images.  Also  new  starting  points  for  paths  are  generated. 

4.2.4  Reducing  the  time  interval 

With  increasing  time  interval  between  two  images,  the  number  of  candidate 
objects  for  a  path  increases  (par  4.2.2).  On  average,  the  number  is  proportional  to 
the  square  of  the  time  interval.  The  radar  determines  the  minimum  length  of  the 
time  interval,  which  equals  the  minimum  time  required  for  the  antenna  to  complete 


2 


A  pathsegment  is  defined  as  the  connection  between  two  objects  in  two  successive 
images. 
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one  rotation.  Because  objects  can  disappear  from  an  image  due  to  effects  described 
in  par.  2.1,  it  is  necessary  to  use  several  rotations  to  be  able  to  bridge  the  gap  in  the 
path  caused  by  the  absence  of  an  object. 

Reducing  the  number  of  time  intervals  between  pairs  of  images,  will  speed  up  the 
matching  process,  but  it  will  cause  some  paths  to  split  up  or  to  disappear  entirely. 

4.2.5  Pruning  of  the  search  tree 

In  the  score  function,  the  score  of  a  path  is  increased  when  there  is  large  stability 
in  the  velocity  vector.  If  large  changes  in  the  velocity  vector  occur  in  a  path,  score 
calculation  can  be  omitted  and  all  extended  paths  do  not  have  to  be  evaluated.  Not 
all  bird  motions  are  equally  regular,  those  paths  associated  with  less  regular  bird 
motions  cannot  always  be  found. 

4.2.6  Optimizing  the  datamodel 

A  path  can  be  regarded  as  a  sequence  of  objects  or  as  a  sequence  of  object  pairs.  In 
the  latter  case  many  intermediate  values  (such  as  Vx;~  +  Vyj^)  may  be  calculated 
once  and  used  many  times.  The  consequence  is  that  a  large  amount  of  data  has  to 
be  stored. 


4.3  The  pathfmding  algorithm 

4.3.1  Preprocessing 

For  every  image  in  the  sequence  a  detection  list  is  generated.  A  detection  list 
consists  of  one  record  for  each  object  in  the  image,  storing  the  location  of  the 
object  (centre  of  gravity),  the  size  (number  of  pixels)  and  mass  (sum  of  the  pixel 
intensities)  of  the  object. 

4.3.2  Initialisation 

From  the  detection  lists  a  tree  is  built.  An  object  is  connected  to  every  other  object 
in  later  images  for  wich  the  path  segment  satisfies  certain  conditions.  So  there  are 
also  segments  that  skip  a  number  of  images. 

Paths  that  already  (partially)  exist  are  treated  in  a  different  way,  each  object  is  only 
linked  to  the  segment  that  is  actually  part  of  the  path.  The  starting  point  of  this 
path  is  connected  to  objects  in  earlier  images,  the  endpoint  of  this  path  can  be  the 
starting  point  of  a  path  segment. 

4.3.3  Pathfinding 

The  segments  are  accessed  in  the  order  of  a  depth-first  tree.  If  a  node  is  reached 
where  no  more  acceptable  continuations  can  be  found,  the  search  continues  at  the 
previous  level.  For  every  candidate  extension  of  the  path  the  score  is  calculated 
from  the  current  path  and  its  extension. 

For  every  segment  the  new  score  is  compared  to  its  previous  best  score.  If  the  new 
score  is  better,  the  score  of  every  segment  of  the  path  is  replaced  by  the  new  score 
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and  a  link  to  the  next  segment  in  the  path  is  stored.  The  first  and  last  path  segments 
are  marked. 

4.3.4  The  administration  of  paths 

Paths  that  have  a  constant  score  from  start  to  end  mark  are  put  in  a  list,  found- 
list.  Objects  that  are  part  of  a  path  in  the  found-list  cannot  be  part  of  any  other 
path,  therefore  these  objects  and  all  segments  that  start  from  or  end  at  these  objects 
must  be  removed  from  the  tree  of  path  segments.  Also  objects  that  are  not  part  of 
any  path  (loose  objects)  are  removed,  becaused  in  the  next  cycle  they  will  again 
turn  up  as  loose  objects.  All  remaining  segments  in  the  tree  are  cleared  and  a  new 
cycle  begins.  This  repeats  until  there  are  no  more  objects  available. 
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5  Conclusions 


The  algorithm  is  easily  adapted  for  more  object  feature  parameters  or  3 
dimensional  data. 

The  algorithm  can  be  run  in  parallel  on: 

•  sub  sequences  of  images; 

•  partial  images; 

•  different  speed  and/or  direction  regions. 

One  extra  run  must  be  made  on  the  whole  sequence  of  complete  images  to  correct 
for  border  effects  (using  the  partial  results). 

The  algorithm  can  be  used  iteratively  in  the  same  way  as  in  parallel  execution,  a 
special  case  is  the  incremental  use  for  continuous  tracking. 

An  implementation  of  the  algorithm  has  been  made  containing  all  optimizations  as 
mentioned  in  4.2  and  iteration  over  speed/direction  regions.  These  regions  are 
based  on  a  cluster  analysis  of  all  possible  velocity  vectors  of  the  remaining  objects 
and  selected  in  a  decreasing  order  of  bird  motion  magnitude  in  the  regions. 

This  implementation  is  in  operational  use  in  the  Royal  Netherlands  Airforce  at  the 
moment. 

The  results  of  this  implementation  in  a  clutter  free  situation  agree  well  with  the 
visual  impression  of  a  motion  picture  of  the  images.  Only  a  few  percents  of  the 
tracks  are  wrong  or  not  found  in  images  with  medium  bird  density.  At  high 
density,  where  the  nearest  neighbour  distance  is  less  than  the  distance  travelled  in 
the  time  between  two  images,  the  eye  is  not  able  to  identify  individual  tracks,  only 
a  global  impression  of  motion  exists.  In  this  case  no  comparison  can  be  made  if  no 
other  means  of  finding  the  true  tracks  are  available. 

If  clutter  is  present  it  is  excluded,  based  on  statitics  in  time  and  space.  Speckled 
moving  rain  regions  are  not  completely  removed  and  the  algorithm  finds  spurious 
tracks  on  the  edges. 

It  is  recommended  to  do  some  more  research  to  avoid  these  tracks  either  by 
improvement  of  the  clutter  filters  and  bird  detection  or  by  postprocessing  of  the 
tracks  found. 
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Appendix  A  Formulae 


The  score  in  [4.1.2]  is  written  as: 


X-(n,P)-XF 


i=l 


Suppose  we  have  k+1  objects,  numbered  0..k.  Because  velocity  is  calculated  from 
the  object  positions,  there  are  k  velocity  vectors  each  having  2  degrees  of  freedom. 
Furthermore  there  are  k+1  object  masses  (see  4.3.1)  each  with  1  degree  of 
freedom. 

The  score  function  in  some  more  detail  is: 


i=l  i=l  i=0 


hi-|4h 


(1) 


where  V  refers  to  speed,  h  to  direction  and  m  to  mass. 


If  the  variation  in  direction  is  relatively  small  then 


a. 


can  be  approximated  by: 

(2viik,^.K2VliL)= 

<5,  O, 

where  Vr  is  the  velocity  component  in  the  long  term  flight  direction  and  Vt  the 
component  perpendicular  to  Vr. 


(2) 


Equation  (2)  is  rewritten  as: 


^  ( Vt,  -  n,  f  -K  Vr,  -  ^  ^ 

^  a~  o/ 


(3) 


TNO  report 


FEL-94-A177 
Appendix  A 


A.2 


The  function 

(Vti-|i,)'+(Vr. 

equals 

where  Vxj  and  Vyj  are  the  velocity  components  in  the  x,  y  coordinate  grid. 


The  last  term  in  (3)  contains  a  factor 


K 


(4) 


If  Oj- «  P-r  and  the  distribution  of  Vr^^  is  approximatily  a  Gaussion  distribution 
with  mean  and  deviation  the  factor  (4)  is  equivalent  to 


2n,a, 

where  equals  Vx;-  +  Vyj^. 


Combining  these  results,  the  score  function  becomes: 

X(VX|-n.)-  +  (Vyi-n,)-  + 


x"(3k  +  t,P)- 


■K 


m;  - 


4)1, -a;- ( 


a~  -a. 


-) 


a„ 


(5) 

The  values  of  (ij,  p,-  and  are  in  general  not  known  beforehand  so  they  must  be 
estimated  from  the  data. 

If  there  are  n  data  values  xj  then  u  is  replaced  by 


denoted  by  <x>,  so 

-2x-  <  X  >  +  <  X  >■)  = 

_  i  =  l _ 

n  (6) 

While  <x>  depends  on  xj  one  degree  of  freedom  is  lost.  This  is  clearly 
demonstrated  by  n  =  1;  formula  (6)  yields  0  in  that  case. 


]^(x -<x>)-  =£(x.' 


i=l 


u  ■» 

Z'>  2  'V’  2 

xf-n<x>  =2^x, 
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In  formula  (5)  three  independent  quantities  are  present,  the  total  degrees  of 
freedom  must  therefore  be  reduced  by  3. 

Substitution  of  (6)  in  (5)  and  the  assumption  that  0^"  =  •  Itm  yields: 

k  k 

^  (ZVXir-  +  (£vy,)^- 

X(Vx,-  +  Vy.-)-^^^ - - 

score  =  x"(3k-2,  P)  -  - - - 


k.X(Vx;-+Vy3-)^-  ^ 

- £(Vxr  +Vy,^) 

£(Vx,-+Vy,-)  ‘=' 


(k  +  l).^m.-  , 

- - 


c 


c 


m 


where:  c,  =  a^^  c^  =4a/.( — ~ — 7)  and  c^  -C7„, 

a,- -a/ 

(7) 

A  minimum  number  of  three  objects  in  a  path  is  necessary  to  calculate  (7). 

The  assumption  of  <3^  =  0^1  •  M-m  b^sed  on  the  physical  properties  of  radar 
reflection.  This  and  the  Gaussian  distribution  of  m;  are  crude  approximations. 
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